Mastering Data-Intensive Collaboration and Decision Making

ion level. In both tasks, the participants were asked to start browsing from an entity classified at a classification low level, i.e. a concrete musical instrument (bouzouki and electric guitar, respectively). Participants clicked much more on entities from instrument classifications in Task 1 than in Task 2 (high significance, Wilcoxon test, W=74, p<0.005). Similarly, participants clicked much more on content entities in Task 2 than in Task 1 (high significance, Wilcoxon test, W=-66, p<0.005), the average click distribution is given in Table 8. The result is not surprising; tasks were set expecting such user browsing behaviour. Further analysis of the log files showed some unexpected cases outlined below. Table 7: Abstraction level assigned to the user clicks Abstraction level Descriptionion level Description Classification – upper level Clicks on abstract concepts, such as instrument, performance, artist, from the Music Ontology Classification – middle level Clicks on categories for musical instruments, e.g. string instruments, plucked string instruments, steel guitars, Greek musical instruments Classification – low level Clicks on entities representing musical instruments, e.g. bouzouki, mandolin, lute, electric guitar. Content albums Clicks on descriptions of music albums Content artists Clicks on descriptions of music artists Content reviews Clicks on Amazon reviews of musical instruments FP7-ICT-2009-5 257184 Dicode D5.3.2 (version 6): Page 27 of 32 Figure 11: Distribution of user clicks in both tasks (total number of clicks in each abstraction level). Confusion of abstract concepts. Two participants clicked on abstract concepts, such as instrument, performance and performer, from the Music Ontology. In both cases, the participants were looking for concrete information (e.g. participant-12 clicked on instrument in Task 1 when seeking for more detail about a musical instrument, while participant-05 clicked on performer and performance in Task 2 when seeking more detail about an album). Clicking on abstract concepts led to confusion, and the participants quickly pressed the back button on their browsers. This implies that it would be beneficial to avoid showing links at high abstract level. Existence of ‘empty clicks’. Another interesting case is the high number of ‘empty clicks’ the user clicks on a link and is taken to a page with no information, sees that this link is not helpful and quickly returns to the previous page. In Task 1, such clicks concerned similar instruments, e.g. there was no information about bajitar, xalam, rebab. In Task 2 such clicks concerned performances (music albums) and happened quite often. ‘Empty clicks’ leading to pages with no information was seen as one of the key usability problems. Some of the links, however, were perceived as empty and the users missed to click on important information (e.g. pages about musical instruments were abandoned, although there was useful information about relevant instruments; or interesting facts about an album artist were overlooked as the users did not click on the corresponding link). A possible way to prevent such problems is to reduce the number of links shown to the user (which will avoid clutter and confusion) and to signpost links with a certain browsing value. Table 8: Distribution of user clicks in both tasks (average per individual participants’ clicks) Selection strategies. Both tasks (deliberately) put the users in situations where they had too many choices. The bouzouki page included 12 different links in the facts facet (11 links to concepts in the middle classification level and 1 link to the abstract concept instrument) and 51 links in the terms facet (43 links to musical instruments and 8 links to performances). FP7-ICT-2009-5 257184 Dicode D5.3.2 (version 6): Page 28 of 32 There were four main strategies observed when selecting a link from the bouzouki page: (i) click on the nearest classification link (e.g. plucked string instruments or string instruments) to see general characteristics; (ii) click on instruments mentioned in the description – lute and mandolin; (iii) click on a instrument that ‘sounds familiar’ (e.g. sitar, banjo, pipa); (iv) click on something that sounds interesting or unusual (e.g. oud, xalam). The electric guitar page included 18 links in the facts facet (to concepts in the middle and upper classification levels), 78 links to albums in the terms facet, and 8 links to Amazon reviews in the content facet. The users rarely clicked on links from the facts facet (as the task did not require this), and adopted some strategies for selecting from the list of albums: (i) clicking on something that looks important (e.g. an artist has several albums); (ii) clicking on something that is interesting or unusual (e.g. noticing a women artist or something interesting in the album name); (iii) clicking at random (which happened after exhausting the first two strategies). The observations about participants’ behaviour with MusicPinta provide useful information about possible improvement of the browser by adding signposting techniques. 3.5.3 Usability Feedback As MusicPinta is still at a prototype level, its usability hinders performance at times. The answers to the usability questionnaire indicated key usability issues that have to be addressed. The main criticism stems from lack of content after clicking on a link (e.g. for electric guitar, 12 out of 25 artists linked to electric guitar had no extra information; only 3 artists had extra links to other albums; 6 artists had biographical information and photos; there was no extra information on content of albums e.g. tracks, and there was no way to listen what the album included). Participants were overwhelmed by the long list of links and baffled by link duplications (when showing the same concept from DBTune and DBpedia). There is evidence that small font text used for description was easily missed by participants despite usefulness of content and position of text. It took a while to get familiar with the terminology and navigate to useful information (as seen from performance in Task 2 when it was done before Task 1). Slow system performance at times was an issue (only one participant did not have issues with performance, as he regularly opened new windows when clicking on browser links). These usability issues will be addressed in the next MusicPinta version.